Generating haptic effects for dynamic events

ABSTRACT

A system that generates a dynamic haptic effect for a dynamic event receives a first endpoint and a second endpoint for dynamic events. The first endpoint includes a first endpoint value and a corresponding first haptic effect, and the second endpoint includes a second endpoint value and a corresponding second haptic effect. The system receives a dynamic value for the dynamic event. The dynamic value is between the first endpoint value and the second endpoint value. The system then determines the dynamic haptic effect from the dynamic value by interpolating the dynamic haptic effect from the first haptic effect and the second haptic effect.

FIELD

One embodiment is directed generally to haptic effects, and inparticular to generating haptic effects in response to a dynamic event.

BACKGROUND INFORMATION

Electronic device manufacturers strive to produce a rich interface forusers. Conventional devices use visual and auditory cues to providefeedback to a user. In some interface devices, kinesthetic feedback(such as active and resistive force feedback) and/or tactile feedback(such as vibration, texture, and heat) is also provided to the user,more generally known collectively as “haptic feedback” or “hapticeffects”. Haptic feedback can provide cues that enhance and simplify theuser interface. Specifically, vibration effects, or vibrotactile hapticeffects, may be useful in providing cues to users of electronic devicesto alert the user to specific events, or provide realistic feedback tocreate greater sensory immersion within a simulated or virtualenvironment.

Haptic feedback has also been increasingly incorporated in portableelectronic devices, referred to as “handheld devices” or “portabledevices”, such as cellular telephones, personal digital assistants(“PDA”s), smartphones, and portable gaming devices. For example, someportable gaming applications are capable of vibrating in a mannersimilar to control devices (e.g., joysticks, etc.) used withlarger-scale gaming systems that are configured to provide hapticfeedback. Additionally, devices such as cellular telephones andsmartphones are capable of providing various alerts to users by way ofvibrations. For example, a cellular telephone can alert a user to anincoming telephone call by vibrating. Similarly, a smartphone can alerta user to a scheduled calendar item or provide a user with a reminderfor a “to do” list item or calendar appointment. Further, haptic effectscan be used to simulate “real world” dynamic events, such as the feel ofa bouncing ball in a video game.

SUMMARY

One embodiment is a system that generates a dynamic haptic effect for adynamic event. The system receives a first endpoint and a secondendpoint for dynamic events. The first endpoint includes a firstendpoint value and a corresponding first haptic effect, and the secondendpoint includes a second endpoint value and a corresponding secondhaptic effect. The system receives a dynamic value for the dynamicevent. The dynamic value is between the first endpoint value and thesecond endpoint value. The system then determines the dynamic hapticeffect from the dynamic value by interpolating the dynamic haptic effectfrom the first haptic effect and the second haptic effect.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a block diagram of a haptically enabled system inaccordance with one embodiment of the invention.

FIG. 2 illustrates an example of a dynamic event that generates a forcefor which a haptic effect is generated.

FIG. 3 illustrates the typical interaction between a haptic effectdesigner and a haptic effect programmer when creating haptic effectsthat reflect a collision of the ball against the wall of FIG. 2, orother dynamic events, in accordance with one embodiment.

FIG. 4 is a flow diagram of the functionality of the haptic effectsgeneration module of FIG. 1 when using interpolation to automaticallygenerate haptic effects for dynamic events in accordance with oneembodiment.

DETAILED DESCRIPTION

One embodiment is a system that generates haptic effects for “dynamicevents” such as a simulated bouncing ball. The system receives thedesired haptic effects for the “endpoints” of the dynamic event, such asthe minimal and maximum force when the ball contacts a wall. The systemthen uses interpolation to automatically generate haptic effects fordynamic events that fall between the endpoints.

Devices that incorporate haptic effects generally are developed with thecooperation of both haptic effect designers, who determine what thehaptic effects should “feel” like, and haptic effect programmers thatdevelop software code to implement the designed haptic effects. In manysystems, an application programming interface (“API”) separates the workof the designer from the programmer so that a designer can call adesired haptic effect by name, and the API retrieves the correspondingcode or routine to implement the desired haptic effect. One example ofan API for haptic effects is the “VibeTonz” API from Immersion Corp.

Haptic effects are frequently used to simulate “dynamic” real worldevents. For example, a video game may feature a ball bouncing off of awall. Depending on the speed/force of the ball against the wall, ahaptic effect that simulates the collision of a bounce must to be variedto reflect the force that would have been generated by the collision inthe real world. The haptic effect can be varied by changing parameters.As another example, a smartphone may display a scrolling list ofcontacts. As the list scrolls, a haptic effect may generate a “tick”haptic effect feel between contacts. As the speed of the scrollingincreases, the tick should get stronger to reflect the increased speed,and vice versa. In one embodiment, when a haptic effect is vibratory andis generated by an actuator, the haptic effect can be varied to simulatedynamic haptic events by varying any combination of magnitude, frequencyand duration of the vibration parameters. Other examples of dynamicevents that can generate corresponding haptic effects include the forceof a boxing glove hitting a person, the force of a bat hitting a ballthe force of a car colliding with another object, etc.

For many simulations, the number of dynamic events for which acorresponding haptic effect is generated can be fairly large. Forexample, for the ball bouncing against the wall, a video game mayspecify ten or more different forces generated by the ball against thewall depending on the speed of the ball. Most designers, in designinghaptic effects for these forces, will merely specify the parameters forthe endpoints (i.e., the smallest force and the largest force). Theprogrammer then must program all parameters in between the endpointsusing linear mapping or some other method. Depending on the number ofintermediate points, this may require a large effort on behalf of theprogrammer. In contrast, embodiments of the present inventionautomatically generate intermediate stage haptic effects based on theendpoints using interpolation.

FIG. 1 illustrates a block diagram of a haptically enabled system 10 inaccordance with one embodiment of the invention. In one embodiment,system 10 is part of a mobile device, and system 10 provides hapticeffect generation for the mobile device. Although shown as a singlesystem, the functionality of system 10 can be implemented as adistributed system, and can generate the haptic effects on system 10itself, or send haptic effects signals or data to another device whichthen generates the haptic effects.

System 10 includes a bus 12 or other communication mechanism forcommunicating information, and a processor 22 coupled to bus 12 forprocessing information. Processor 22 may be any type of general orspecific purpose processor. System 10 further includes a memory 14 forstoring information and instructions to be executed by processor 22.Memory 14 can be comprised of any combination of random access memory(“RAM”), read only memory (“ROM”), static storage such as a magnetic oroptical disk, or any other type of computer-readable medium.

A computer-readable medium may be any available medium that can beaccessed by processor 22 and may include both a volatile and nonvolatilemedium, a removable and non-removable medium, a communication medium,and a storage medium. A communication medium may include computerreadable instructions, data structures, program modules or other data ina modulated data signal such as a carrier wave or other transportmechanism, and may include any other form of an information deliverymedium known in the art. A storage medium may include RAM, flash memory,ROM, erasable programmable read-only memory (“EPROM”), electricallyerasable programmable read-only memory (“EEPROM”), registers, hard disk,a removable disk, a compact disk read-only memory (“CD-ROM”), or anyother form of a storage medium known in the art.

In one embodiment, memory 14 stores software modules that providefunctionality when executed by processor 22. The modules include anoperating system 15 that provides operating system functionality forsystem 10, as well as the rest of a mobile device in one embodiment. Themodules further include a haptic effects generation module 16 thatgenerates haptic effects using interpolation, as disclosed in moredetail below. System 10 will typically include one or more additionalapplication modules 18 to include additional functionality, such assmartphone related applications (if system 10 is a smartphone), APIs, aphysics system, etc. System 10 may further be coupled to a database 30for storing data used by modules 16 and 18.

System 10, in embodiments that transmit and/or receive data from remotesources, further includes a communication device 20, such as a networkinterface card, to provide mobile wireless network communication, suchas infrared, radio, Wi-Fi, or cellular network communication. In otherembodiments, communication device 20 provides a wired networkconnection, such as an Ethernet connection or a modem.

Processor 22 is further coupled via bus 12 to a display 24, such as aLiquid Crystal Display (“LCD”), for displaying a graphicalrepresentation or user interface to a user. The display 24 may be atouch-sensitive input device, such as a touchscreen, configured to sendand receive signals from processor 22, and may be a multi-touch touchscreen. Processor 22 is further coupled to a keyboard or cursor control28, such as a mouse, that allows a user to interact with system 10.

System 10, in one embodiment, further includes an actuator 26. Processor22 may transmit a haptic signal associated with a generated hapticeffect to actuator 26, which in turn outputs haptic effects such avibrotactile haptic effects. Actuator 26 includes an actuator drivecircuit. Actuator 26 may be, for example, an electric motor, anelectro-magnetic actuator, a voice coil, a shape memory alloy, anelectro-active polymer, a solenoid, an eccentric rotating mass motor(“ERM”), a linear resonant actuator (“LRA”), a piezoelectric actuator, ahigh bandwidth actuator, an electroactive polymer (“EAP”) actuator, anelectrostatic friction display, or an ultrasonic vibration generator. Inother embodiments, a separate device from system 10 includes an actuatorthat generates the haptic effects, and system 10 sends generated hapticeffect signals to that device through communication device 20.

FIG. 2 illustrates an example of a dynamic event that generates a forcefor which a haptic effect is generated. In FIG. 2, a ball hits one offour walls. Each collision with a wall generates a “force” that variesdepending on the speed of the ball. The “ball” and “walls” of FIG. 2 canbe part of a video game shown on display 24 of system 10 of FIG. 1 inone example. A “physics system” is part of the dynamic event anddetermines a real world equivalent force, velocity, etc. associated withthe dynamic event. The physics system can be computer software thatprovides an approximate simulation of certain physical systems, such asrigid body dynamics (including collision detection), soft body dynamics,and fluid dynamics. In other embodiments, rather than generating aphysics system based dynamic event, the input may be, for example, timein an animation where the force changes as a function of time.

FIG. 3 illustrates the typical interaction between a haptic effectdesigner 32 and a haptic effect programmer 44 when creating hapticeffects that reflect a collision of the ball against the wall of FIG. 2,or other dynamic events, in accordance with one embodiment. Designer 32designs the “feel” of the haptic effect that corresponds to eachcollision. Designer 32 specifies the endpoint range of vibrationparameters (e.g., minimum and maximum values Vmin 35 and Vmax 37 forstrength and periodicity) based on knowledge of the feeling of thetarget device. The vibration limits 36 reflect the endpoints of therange and specify signals for actuator 26 to generate haptic effects forthe smallest force, and haptic effects for the largest force. Theendpoints may not be the “absolute” endpoints of a range. Instead, the“endpoints” may be intermediate endpoints between two additional valuesthat may also be considered endpoints. For example, for a range of 1-10,the endpoints chosen may be 3 and 7.

Programmer 44 programs processor 22 of FIG. 1 so that the haptic effectsignals are generated. Programmer 44 specifies a range of force inputs(e.g., minimum and maximum values Fmin 45 and Fmax 47) based onknowledge of the range of typical values from the physics system orsimulation. The collision limits 46 reflect the minimum and maximumforce that would be generated by the ball hitting a wall.

In operation, embodiments calculate the collision force of a particulardynamic event that may fall between the endpoints (i.e., not thesmallest force and not the largest force). The output to actuator 26 isinterpolated from the programmer's collision limits 46 to the designer'svibration limits 36.

FIG. 4 is a flow diagram of the functionality of haptic effectsgeneration module 16 of FIG. 1 when using interpolation to automaticallygenerate haptic effects for dynamic events in accordance with oneembodiment. In one embodiment, the functionality of the flow diagram ofFIG. 4 is implemented by software stored in memory or other computerreadable or tangible medium, and executed by a processor. In otherembodiments, the functionality may be performed by hardware (e.g.,through the use of an application specific integrated circuit (“ASIC”),a programmable gate array (“PGA”), a field programmable gate array(“FPGA”), etc.), or any combination of hardware and software.

At 402, the next dynamic event is detected. The functionality of FIG. 4is executed as a continuous loop so a dynamic event is always detected.

At 404, it is determined if the dynamic event (e.g., a collision of aball into a wall) happened. If No at 404, the functionality continues at402. If Yes at 404, the physic systems provides a dynamic event value.In the example of FIG. 2, the event is the collision, and the dynamicevent value is the collision force (e.g., 8 on a scale of 1-10). Thedynamic event value is passed to a routine in one embodiment called“playDynamicEffect.” In one embodiment, the dynamic event value ispassed to the routine as follows:

playDynamicEffect(“collision”, force); where ‘collision” is the name ofthe haptic event, and the force is the value of the haptic event.

For example, assume the following effects are defined in an effectsfile:

Name=“collision 1”, Duration=10 ms, Magnitude=0, Period=5 ms;

Name=“collision 10”, Duration=50 ms, Magnitude=10000, Period=5 ms.

In this example, the haptic effects are defined by three parameters:duration, magnitude, and period.

At 406, the set of effect definitions closest to the dynamic event valueare retrieved from the effects file. A linear interpolation in oneembodiment requires at least two definitions. In the above example,these effects are found by their common name “collision”, and include“collision 1” and “collision 10”. More than two effects can be definedand retrieved from the effects files. For example, the following threeeffects may be defined in an effects file:

Name=“collision 0.25”, Duration=10 ms, Magnitude=0, Period=5 ms;

Name=“collision 1”, Duration=10 ms, Magnitude=0, Period=5 ms;

Name=“collision 10”, Duration=50 ms, Magnitude=10000, Period=5 ms.

At 408 and 410, a determination is made as to whether the dynamic eventvalue falls between the two definition values. For example, module 16determines that “8” is between the “1” and “10” encoded in the names ofthe definition values.

If the dynamic event value is greater than the lowest set definitionvalue at 408, in one embodiment no effect is played and thefunctionality continues to 410. This provides for a “deadband” whereinput forces are ignored. In another embodiment, the smallest effectdefinition in the set may be used and the functionality then returns to402.

If No at 408, if the dynamic event value is less than the greatest setdefinition value at 410, the highest effect definition in the set isused at 412 (e.g., 10) and the functionality returns to 402.

If Yes at 410, at 414 the haptic effect definition is determined byinterpolation at 414 and the functionality returns to 402.

In order to determine the haptic effect by interpolation at 414, in oneembodiment an interpolation variable “t” is determined using thefollowing:t=(dynamic event value−lowest value dynamic event)/(highest valuedynamic event−lowest value dynamic event)For the above example,

t=(8−1)/(10−1), so t is approximately equal to 0.78.

In a special case when the highest value dynamic event equals the lowestvalue dynamic event, t=1 to avoid division by zero.

In one embodiment, t is used in the following linear interpolationfunction:(1−t)*A+t*Bto calculate each interpolated haptic effect parameter, where “A” is theparameter value for the lowest value dynamic event, and “B” is theparameter value for the highest value dynamic event. For the above twoeffects example, the parameters are determined as follows:Duration=(1−t)*10+t*50=41 ms;Magnitude=(1−t)*0+t*10000=7778;Period=(1−t)*5+t*5=5 ms.Therefore, after 414, the interpolated haptic parameter values (i.e., 41ms duration, 7778 magnitude, 5 ms period) are output to actuator 26 ofFIG. 1, or another actuator, either indirectly or directly, to cause thehaptic effect to be generated. The functionally then returns to 402 towait for another dynamic event to happen.

In another embodiment, instead of two haptic effect definitions (i.e.,the two endpoints), three haptic effect definitions are used asdescribed above. In this embodiment, the following quadratic equation isused for the interpolation:(1−t)^2*A+2*(1−t)*t*B+t^2*C,where C is the third definition.

As disclosed, embodiments allow a haptic effect designer to more easilyimplement rich haptic effects by specifying the desired haptic effectendpoints. Embodiments then automatically generate intermediate stagehaptic effects through interpolation.

Several embodiments are specifically illustrated and/or describedherein. However, it will be appreciated that modifications andvariations of the disclosed embodiments are covered by the aboveteachings and within the purview of the appended claims withoutdeparting from the spirit and intended scope of the invention.

What is claimed is:
 1. A non-transitory computer readable medium havinginstructions stored thereon that, when executed by a processor, causethe processor to generate a dynamic haptic effect for a dynamic event,the generating the dynamic haptic effect comprising: receiving a firstendpoint and a second endpoint for dynamic events, wherein the firstendpoint comprises a first endpoint value and a corresponding firsthaptic effect, and the second endpoint comprises a second endpoint valueand a corresponding second haptic effect, wherein the first hapticeffect comprises a plurality of first parameters and the second hapticeffect comprises a plurality of second parameters; receiving a dynamicvalue for the dynamic event, wherein the dynamic value is between thefirst endpoint value and the second endpoint value; and determining thedynamic haptic effect from the dynamic value, wherein the determiningcomprises interpolating the dynamic haptic effect from the first hapticeffect and the second haptic effect; wherein the interpolating comprisesgenerating an interpolation value (t) that is a function of the firstendpoint value, the second endpoint value, and the dynamic value;wherein the interpolation value (t) comprises: (dynamic eventvalue−first endpoint value)/(second endpoint value−first endpointvalue); wherein the first endpoint value is a lowest endpoint value, andthe second endpoint value is a highest endpoint value; and wherein theinterpolating further comprises, for each parameter of the dynamichaptic effect, calculating each parameter as a function of theinterpolation value (t), a corresponding first haptic effect parameter,and a corresponding second haptic effect parameter.
 2. Thenon-transitory computer readable medium of claim 1, wherein the dynamichaptic effect is a vibratory haptic effect and comprises a plurality ofparameters.
 3. The non-transitory computer readable medium of claim 2,wherein the plurality of parameters comprise duration, magnitude andperiod.
 4. The non-transitory computer readable medium of claim 2,wherein the vibratory haptic effect is generated by an actuator.
 5. Thenon-transitory computer readable medium of claim 1, wherein the dynamicevent comprises a force.
 6. The non-transitory computer readable mediumof claim 5, wherein the force comprises a simulated object contactinganother simulated object.
 7. The non-transitory computer readable mediumof claim 5, wherein the force is generated by a physics system.
 8. Thenon-transitory computer readable medium of claim 1, wherein theinterpolating comprises, for each parameter of the dynamic hapticeffect, calculating each parameter as (1−t)*A+t*B, wherein A comprisesthe corresponding first haptic effect parameter and B comprises thecorresponding second haptic effect parameter.
 9. A computer-implementedmethod for generating a dynamic haptic effect for a dynamic event, themethod comprising: receiving a first endpoint and a second endpoint fordynamic events, wherein the first endpoint comprises a first endpointvalue and a corresponding first haptic effect, and the second endpointcomprises a second endpoint value and a corresponding second hapticeffect, wherein the first haptic effect comprises a plurality of firstparameters and the second haptic effect comprises a plurality of secondparameters; receiving a dynamic value for the dynamic event, wherein thedynamic value is between the first endpoint value and the secondendpoint value; and determining the dynamic haptic effect from thedynamic value, wherein the determining comprises interpolating thedynamic haptic effect from the first haptic effect and the second hapticeffect; wherein the interpolating comprises generating an interpolationvalue (t) that is a function of the first endpoint value, the secondendpoint value, and the dynamic value; wherein the interpolation value(t) comprises: (dynamic event value−first endpoint value)/(secondendpoint value−first endpoint value); wherein the first endpoint valueis a lowest endpoint value, and the second endpoint value is a highestendpoint value; and wherein the interpolating further comprises, foreach parameter of the dynamic haptic effect, calculating each parameteras a function of the interpolation value (t), a corresponding firsthaptic effect parameter, and a corresponding second haptic effectparameter.
 10. The method of claim 9, wherein the dynamic haptic effectis a vibratory haptic effect and comprises a plurality of parameters.11. The method of claim 10, wherein the plurality of parameters compriseduration, magnitude and period.
 12. The method of claim 10, wherein thevibratory haptic effect is generated by an actuator.
 13. The method ofclaim 9, wherein the dynamic event comprises a force.
 14. The method ofclaim 13, wherein the force comprises a simulated object contactinganother simulated object.
 15. The method of claim 13, wherein the forceis generated by a physics system.
 16. The method of claim 9, wherein theinterpolating comprises, for each parameter of the dynamic hapticeffect, calculating each parameter as (1−t)*A+t*B, wherein A comprisesthe corresponding first haptic effect parameter and B comprises thecorresponding second haptic effect parameter.
 17. A system thatgenerates a dynamic haptic effect for a dynamic event, the systemcomprising: a processor; a memory coupled to the processor and storing ahaptic effects generation module; receiving by the haptic effectsgeneration module a first endpoint and a second endpoint for dynamicevents, wherein the first endpoint comprises a first endpoint value anda corresponding first haptic effect, and the second endpoint comprises asecond endpoint value and a corresponding second haptic effect, whereinthe first haptic effect comprises a plurality of first parameters andthe second haptic effect comprises a plurality of second parameters;receiving by the haptic effects generation module a dynamic value forthe dynamic event, wherein the dynamic value is between the firstendpoint value and the second endpoint value; and determining by thehaptic effects generation module the dynamic haptic effect from thedynamic value, wherein the determining comprises interpolating thedynamic haptic effect from the first haptic effect and the second hapticeffect; wherein the interpolating comprises generating an interpolationvalue (t) that is a function of the first endpoint value, the secondendpoint value, and the dynamic value; wherein the interpolation value(t) comprises: (dynamic event value−first endpoint value)/(secondendpoint value−first endpoint value); wherein the first endpoint valueis a lowest endpoint value, and the second endpoint value is a highestendpoint value; and wherein the interpolating further comprises, foreach parameter of the dynamic haptic effect, calculating each parameteras a function of the interpolation value, a corresponding first hapticeffect parameter, and a corresponding second haptic effect parameter.18. The system of claim 17, further comprising an actuator coupled tothe processor, wherein the actuator outputs haptic effects in responseto receiving the dynamic haptic effect.
 19. The system of claim 18,wherein the haptic effects are vibratory haptic effects.
 20. The systemof claim 17, wherein the interpolating comprises, for each parameter ofthe dynamic haptic effect, calculating each parameter as (1−t)*A+t*B,wherein A comprises the corresponding first haptic effect parameter andB comprises the corresponding second haptic effect parameter.
 21. Anon-transitory computer readable medium having instructions storedthereon that, when executed by a processor, cause the processor togenerate a dynamic haptic effect for a dynamic event, the generating thedynamic haptic effect comprising: receiving a first endpoint and asecond endpoint for dynamic events, wherein the first endpoint comprisesa first endpoint value and a corresponding first haptic effect, and thesecond endpoint comprises a second endpoint value and a correspondingsecond haptic effect; receiving a dynamic value for the dynamic event,wherein the dynamic value is between the first endpoint value and thesecond endpoint value; and determining the dynamic haptic effect fromthe dynamic value, wherein the determining comprises interpolating thedynamic haptic effect from the first haptic effect and the second hapticeffect; wherein the interpolating comprises generating an interpolationvalue (t) comprising: (dynamic event value−first endpoint value)/(secondendpoint value−first endpoint value); and wherein the first endpointvalue is a lowest endpoint value, and the second endpoint value is ahighest endpoint value.
 22. The non-transitory computer-readable mediumof claim 21, wherein the first haptic effect comprises a plurality offirst parameters and the second haptic effect comprises a plurality ofsecond parameters; wherein the interpolating comprises, for eachparameter of the dynamic haptic effect, calculating each parameter as(1−t)*A+t*B, wherein A comprises a corresponding first haptic effectparameter and B comprises a corresponding second haptic effectparameter.